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Problems with Automation Ss 


e Brittle 


— Automation often operates well for a range of situations but requires human 
intervention to handle boundary conditions (Woods & Cook, 2006) 


e Opaque 
— Automation interfaces often do not facilitate understanding or tracking of the system 
(Lyons, 2013) 


Miscalibrated Trust 


— Disuse and misuse of automation have lead to real-world mishaps and tragedies (Lee 
& See, 2004; Lyons & Stokes, 2012) 


¢« Out—of-the-Loop Loss of Situation Awareness 
— Trade-off: automation helps manual performance and workload but recovering from 
automation failure is often worse (Endsley, 2016; Onnasch, Wickens, Li, Manzey, 
2014) 


HAT Solutions to Problems with Automation Ss 


e Brittle 
— Negotiated decisions puts a layer of human flexibility into system behavior 


e Opaque 
— Requires that systems be designed to be transparent, present rationale and 
confidence 
— Communication should be in terms the operator can easily understand (shared 
language) 


Miscalibrated Trust 
— Automation display of rationale helps human operator know when to trust it 


¢ Out—of-the-Loop Loss of Situation Awareness 
— Keep operator in control; adaptable, not adaptive automation 


— Greater interaction (e.g., negotiation) with automation reduces likelinood of being out 
of the loop 


Simulated Ground Station 


= 


Autonomous Constrained Flight Planner (ACFP) 


GS 394 1AS237 (As: 160 ww | 
KPUB,RW26L ALT FL410 1:06:44; 
sate ea olteifarsy 


KGJT,RW29 


KCYS,RW27 


KPUB,RWO08R,SP 


KGJT,RW11,SP 


KEGE,RW25,XW,SP 


KCOS,RW35R,XW 


KCOS,RW35L,XW 


Recommended airports 
- rank ordered. 


NASA12 
FL410 434 


Adding HAT Principles to the Ground Station 


With Added 
Transparency 


KLAX (251) 


V KLAX (251) 


WATIs 
29015G30KT 1SM RA OVC011 BKNO21 20/18 29.98 (TWO 


NINE NINE EIGHT). NO PUBLISHED APPROACH... ADVS YOU 
HAVE INFO F 
KLAX, 25L, 11095, ILS25L, 92.51 


YW Path Rating: Marginal 
P ENROUTE: Acceptable 
DP APPROACH: Acceptable 
V RUNWAY: Marginal 


The runway crosswind conditions are marginal for landing. 
The runway width, the length, the speed because of the 
tailwind component, and the surface are acceptable for 
landing. 


KSAN (27) 
KLAX (25L) | Reset Execute 
| of | | Enter ff Undo CC 


Adding HAT Principles to the Ground Station 


NASAO4 
NASA100 
NASA106 
NASA111 
NASA112 
NASA113 
NASA114 
NASA138 
NASA139 
NASA143 
NASA147 
NASALS 
NASA159 
NASA161 
NASA165 
NASA166 
NASA168 
NASA20 
NASA24 
NASA38 
NASA39 
NASA4S 
NASA46 
NASAG3 
NASA76 
NASA82 
NASAS3 


NASA86 


ABQ ~ DEN 
ONT = GEG 
PHX ~ DEN 
PHX = SEA 
PDX - ABQ 
PDX = COS 
PDX ~ DEN 
SAN -TuS 
SFO - DEN 
SFO - PHX 
SJC= DEN 
DAL - LAX 
SEA-LAS 
SEA ~ PHX 
TUS - DEN 
TUS ~ LAX 
TUS ~ SAN 
DAL~ SEA 
DEN ~ BOI 
DEN - SIC 
DEN - SEA 
BOI ~ OAK 
BOI - PHX 
LAX ~ SLC 
LAS ~ PDX 
LAS = SIC 


LAS ~ SEA 


OAK = ABQ 


BRIEFINGS 


YWNASA35: DEN - SLC; ETA 19:39 

Priority: EMERGENCY 
Filed: DEN./,RLG.EKR.LEEHY3.PLAGEILS34R 

P Alerts: 

D Crew: NOLAN STALLMAN 

P Souls On Board: 88 

DP Equipage: 737-800 

DP Next Waypoint: EKR 


11 


Adding HAT Principles to the Ground Station Ss 


e Transparency: Divert reasoning and ACFP Weights 
factor weights are displayed. 


¢ Negotiation/Dialog: Operators can 


change factor weights to match their 
priorities. <a ————— 


ACFP Recommendations 
Option: KCYS 27 KABQ 08 KABQ 03 KDEN 35L 


e Shared Language/Communication: 


Numeric output from ACFP was found (GE Spaced 

to be misleading by pilots. Display NOW Fee == = aon 
uses English categorical descriptions. 

z= = 

oe. 


NASA HUB 
ike ane ee 
NPs TRAUMA 1M g TRAUMA 3M TRAUMA 3M | TRAUMA 10M 
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Adding HAT Principles to the Ground Station Cy 


e Human-Directed: Operator calls “Plays” to determine who does what 


NASA35 - Medical Emergency 


CO 


C 
OL 
CO 
O 
Cl 
O 
O 
O 
C 
O 


SWITCH STATUS TO MEDICAL 


SUGGEST DIVERT OPTIONS FOR NEAREST SUITABLE 


MAKE RECOMMENDATION TO PILOT 


UPLINK AGREED UPON FLIGHT PLAN 


ADD DETAILS OF ILLNESS TO OPERATOR NOTES 


CONTACT EMS 


CONTACT MAINTENANCE 


CONTACT CUSTOMER SERVICE 


CONTACT SLOT CONTROL 


CONTACT CARGO CONTROL 


ASK IF PILOT NEEDS ADDITIONAL ASSISTANCE 


HAT Simulation: Tasks Ss 


e Participants, with the help of automation, monitored 30 aircraft 
— Alerted pilots when 
e Aircraft was off path or pilot failed to comply with clearances 
e Significant weather events affect aircraft trajectory 
e Pilot failed to act on EICAS alerts 
— Rerouted aircraft when 
e Weather impacted the route 
e System failures or medical events force diversions 


e Ran with HAT tools and without HAT tools 
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HAT Simulation: Results Ss 


e Participants preferred the HAT condition overall (rated 8.5 out of 9). 


¢ HAT displays and automation preferred for Keeping up with operationally 
important issues (rated 8.67 out of 9) 


e HAT displays and automation provided enough situational awareness to 
complete the task (rated 8.67 out of 9) 


e HAT displays and automation reduced the workload relative to no HAT (rated 
8.33 out of 9) 


HAT Simulation: Results Ss 


¢ HAT workload reduction was marginally significant (HAT mean 1.7; No HAT 
mean 2.3, p = .07) 


—™ HAT 

4 “Fe NOHAT 
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ATWIT Probe Number 


HAT Simulation: Debrief Ss 


¢ Transparency/Shared Language 


— “This [the recommendations table] is wonderful.... You would not find a dispatcher 
who would just be comfortable with making a decision without knowing why.” 


e Negotiation 
“The sliders was [sic] awesome, especially because you can customize the route.... | 
am able to see what the difference was between my decision and [the computer’s 
decision].” 


e Human-Directed Plays 


— “This one was definitely awesome. Sometimes [without HAT] | even took my own 
decisions and forgot to look at the QRH because | was very busy, but that didn’t 
happen when | had the HAT.” 
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Where we are and planned FY17 work Ss 


e Trust repair with automated system part-task 
Now (Transparency Part Task) 


e¢ Implementing HAT features on the flight deck 
Spring (17 (Flight Deck) 


e Developing a software framework for creating HAT Agents 
e Updating ground station re-routing tool 


Summer 17 (Ground Station Agent) 
e UX testing 


CHAP-E Ss 
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Hierarchical Activity Planning 


e Abstract idea of what will happen next 
— Abstract plans, not fully defined (instantiated) at start 


e Partially ordered, conditions on tasks 
— Some tasks can be completed in any order 
— Timing is dependent on circumstances 


¢ Precise tasks become more clear as time goes on 
— Interleaved execution and expansion 
— Clearance changes, weather, equipment failures, errors cause plan revision 
— Monitoring/projection detects failures, triggers revision 


Flight (from, to) 


FileFPlan(from, to, alt) ObtainClearance Fly(from,to) Taxi(gate) 


Land(rwy) ' GoAround 


Takeoff(rwy) Approach 


Activity Plan Components Ss 


* Tasks 
— Primitive eae _— 
— Non-primitive 
e Methods = ae 
— Method T: 
Parameters: x,y 
Subtasks: T1, T2, T3, T4 
‘i P * ‘ ec ee 
Constraints/Limitations: T1 -> T3, C -> T3 
e Planner 
| | 
— Expansion of tasks using methods 


— Satisfaction of constraints 


Activity Plan Construction 


Flight Processes Periodic Monitoring / Triggers 


Initial 
Approach 


Clearance Monitoring 


ATC 
Approach 
Clearance 


Periodic 


ae ATC Clearance 
raeyalixeyaiate) 


Airspeed Setting Process Set 
FeV as) ol =1e10 fmt) 
<airspeed> 


Airspeed Monitoring 


Verify speed Call out 
setting <airspeed> 


Periodic 
Fellas) estore | 
(ol g{=101 4 


Periodic 
FeV gs) ol =1e10| PONTE) e1=i=10| 
check 


Clearance Process 


Is airspeed 
within 
(oroyalsiigeliniisiea 


(ooynilaan 
ATC 
(oi(=rele- laters) Is airspeed 
ieere tse) rel) (= 
and within 


(oro)alsiigeliniesiea 


Inform PF: 


ia © =o) ans) 0] =1=10 
No 


ae 
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Activity Plan Construction 


Flight Processes Periodic Monitoring/ Triggers 


Clearance Monitoring 


Airspeed Setting Process 


Airspeed Monitoring 


Approach 


Clearance Process 


CaptureGlideslope SetMissedApprAlt 


Altitude Setting Process 


SetSpeed(VREF+5) SetFlaps(LandingFlaps) 


Altitude Monitoring 


Flap Setting Process 


— (PF) RequestFlaps 
Translate to as 
Hierarchy 5 —— (PM) CheckSpeed 
li (PM) MoveFlaps 
ce Z (PM) CallOutFlaps 


| (PF) ConfirmFlaps 


Localizer Capture Process 


Glideslope Capture Process 


Localizer Monitoring 


Cancel Approach Process 
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Example Activity Plan 


ARCHI 


7000 ft 
DUMBA CEPIN FAF+3 Far+4 AXMUL 


4000 ft 3000 ft 1800 ft 
1000 AGL500 AGL 


Clearance ILS 28R 
Arm Localizer 


Localizer Capture 


CHAN Gr] olde] a=) 
Gear Down Final Approach 


Flaps 5 iq Flaps 20 
oe 


NTs ol=1=10 ak) PANT as o]=1=10 mt k=10) 


4 - aise iy) Replanning ia Future 
Scrolls this direction Required Task Group 


(high level task) 
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Example Activity Plan 


DUMBA CEPIN FAF+3 Far+4 AXMUL 


4000 ft 3000 ft 1800 ft 
1000 AGL500 AGL 59 AGL 


G/S Capture 
Final Checklist 
Arm Speed brakes 


Gear Down 


Flaps 20 
Airspeed 180 Airspeed 135 


Stablize Approach 


‘ - rain i Replanning i Future 
Scrolls this direction Required Task Group 


(high level task) 
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Example Activity Plan 


ARCHI 


7000 ft 
DUMBA CEPIN FAF+3 Far+4 AXMUL 


4000 ft 3000 ft 1800 ft 
1000 AGL500 AGL 


Clearance ILS 28R 
Arm Localizer 


Localizer Capture 


CHAN Gr] olde] a=) 
Gear Down Final Approach 


Flaps 5 iq Flaps 20 
oe 


NTs ol=1=10 ak) PANT as o]=1=10 mt k=10) 


4 - aise Si Replanning i Future 
Scrolls this direction Required Task Group 


(high level task) 
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Example Activity Plan 


ARCHI 


7000 ft 
DUMBA CEPIN FAF+3 Far +4 AXMUL 


4000 ft 3000 ft 1800 ft 
1000 AGL500 AGL50 AGL 


Clearance 270/ +3500 Clearance ILS 28R 


G/S Capture 
Gear Down Final Approach 


Flaps 20 


PNT aj ele=10 16) 


Speedbrakes 


_—_—_—_—_—_—_—_—_—_—_—_—_—_—— - Neminai x Replanning i Future 
Scrolls this direction Required Task Group 


(high level task) 
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Projection 


4000 fit S0D0 ft 1800 


Stochastic 
Sampling 
Arm Localizer & Local Search 


Clearance ILS 28R 


Localizer Capture 


G/S Capture 


Gear Down alate \syelcer-(aa) 


Flaps 5 q Flaps 20 
Fa ARE, 
Airspeed 210 t pirspeed 180 


Fast Time Simulation 


Monitors and Reaction 


e Execution monitors check aircraft situation 
e Remedial actions to repair plan 
e Unplanned Contingencies 


METAR data: 


ee 1. Set flaps 15 
— ae Set missed approach altitude 


2. Set speed 145 
Check radar altimeter 
Sa h §, Set flaps | 
3. Capture localizer at Set speed 130 
Capture gidesibpe =~ " Fun landing checklist 
4, Set flaps 20 a ge oO 
Arm speed breaks a | Eipcocrecsensn F 
———— @. New plan: Approach to 

longer runway 
recommended. 


Testing & Integration 


Flight Simulation Integration with PLEXIL (Oe | 


/ File Run View Debug 


Instrument monitorin Plan Execution 7; Name State Outcome Failure Type | 
_ g / @ [| Main EXECUTING NO_OUTCOME NO FAILURE 
Interchange Language / & % Loop EXECUTING NO_OUTCOME NO _FAILURE 
. 7 ¢ % Procedure EXECUTING NO_OUTCOME NO_FAILURE 
— Automated flight / StartNotify SUCCESS NO_FAILURE 
‘ & % unfreeze SUCCESS NO_FAILURE 
armAP : EC SUCCESS NO_FAILURE 
o &% armAT WAITING NO_OUTCOME NO FAILURE — 
& % engageAT WAITING NO_OUTCOME NO _FAILURE (oe) 
>  %& armLNAV WAITING NO_OUTCOME NO _FAILURE o) 
= & % armvnav WAITING NO_OUTCOME NO_FAILURE 
=  % armFD WAITING NO_OUTCOME NO _FAILURE = 
= & % selectias WAITING NO_OUTCOME NO _FAILURE 
= © @% setAltl800 WAITING NO_OUTCOME NO FAILURE 5) 
= & % armapP WAITING NO_OUTCOME NO _FAILURE _} 
= © BWsetiASl75 WAITING NO_OUTCOME NO_FAILURE 2 
* © @% setFlaps15 WAITING NO_OUTCOME NO _FAILURE 
@ BsetFlaps20 WAITING NO_OUTCOME NO FAILURE <b) 
& &% gearDown WAITING NO_OUTCOME NO FAILURE Q 


© @setFlaps25 WAITING NO_OUTCOME NO_FAILURE 
@ % setiAS1SO —- WAITING NO_OUTCOME NO_FAILURE 
© @%setFlaps30 WAITING NO_OUTCOME NO _FAILURE 
© @ setiAS140 ~—- WAITING NO_OUTCOME NO_FAILURE 
 % setAltitude32(WAITING NO_OUTCOME NO_FAILURE 
& % reverseThrottlWAITING NO_OUTCOME NO_FAILURE 
& % setSpeedBrakWAITING NO_OUTCOME NO_FAILURE 
© BidleThrottle WAITING NO_OUTCOME NO FAILURE 


\\Execution stopped Listening on port 49100 


Transition to 


Ul Prototype UI Design 
— Timeline view 


— Gantt chart style 
¢ Based on location of aircraft 
¢ Timeline of best start times (not 
duration) 
— Matches with trajectory vertical 
profile and waypoints 
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Common Themes Ss 


= Focus on Operational Decision Making 


= Evolution from Pilot Decision Support to Human-Autonomy Teaming 


Explosion of Alert Messages 


Qantas A380 Uncontained Engine Failure 


"QF 32; Singapore to Sydney; 469 people on board 


=4 minutes after Take-off, engine no. 2 bursts, severely 
damaging other equipment 


» 43 ECAM messages in first 60 seconds; 10 additional later 


= 50 minutes to sort through the non-normal checklists (NNCs) 


: SfKFhi set Gs FAUWIts VII 
“It was hard to work out a list of 


what had failed; it was getting to 
be too much to follow. So we 
inverted our logic: Instead of 
worrying about what failed, | 
said ‘Let’s look at what’s 
working.” A380 Captain 


es00e4222e09 8 oe * - 
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Current Approach to Aircraft System Alerting Ss 


Identify urgent actions 
(for stable flight) 


Identify a non-normal 
checklist (NNC) tied &, 


to a component failure Complete NNCs, | 
Not Prioritized as needed Not Organized 
CN by Flight Phase 
Use “Notes” to identify 
Contradictions operational limitations 
» 
Make decision 


about need to divert 


Vo Decision Aid 
/ Support 
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What is a Capability? 


Airplane System Components 


= Hydraulic system 
= Thrust Reverser 
= Battery 


Airplane system 
= Air conditioning pack P y 


components have failed 


Airplane Capabilities 


= Range / Endurance 

= Stopping Distance (on runway) 

= Ability to perform a specific approach 

= Ability to enter RVSM airspace What can | do? 


Where can | go? 


Explicit Alerting on Capabilities 


Typically, we don't 


Can | Fly? 


"78/7 
"449 EICAS messages (Warning, Caution, Advisory) 
= All but 19 of them reflect physical system failures/ 
status changes 


Explicit Alerting on Capabilities 


Sometimes, we do.... 


Examples from the 787 
=" NO AUTOLAND 

=" NO LAND 3 

=" NAV UNABLE RNP 

= STALL PROTECTION 


The New Generation of Systems is Different Ss 


So are the pilots.... 


Airplane System ee Pilot System Knowledge y 


= Airplanes have become more integrated—more shared resources, more 
interconnections—and failures can have effects that are difficult to 
anticipate or understand 


= The volume and rate of crew alert and status messages can increase 
significantly for certain types of failures 


= Non-normal procedure design for combinations of failures is challenging 


= Air turnbacks or diversions occur due to confusion about severity of the 
failures, and impact on the mission 


Both types of errors occur: 
- Poor understanding of real problems 
- Oversensitivity to trivial changes 


Three Types of Information for the Pilot Ss 


Answering Basic Questions 


= Status of Airplane Capabilities 

= What is working/what is not? 

= How can | restore what has been lost? 
=" Operational Guidance 


" Which limitations do | need to observe during the 
remainder of the mission? 


= Mission Objectives 
= Can | still complete the planned mission? 
= If not, where else can | land? 


An Alternative Approach Ss 


Airplane System 
Failure 


Identify urgent actions 
(for stable flight) 


Present an overview of = Sider csiaalia Decision Support for 
= airplane capabilities budliahtoh Mission Decision 
(in addition to EICAS/ECAM) ae 
Goals: operate with an 
Goals: reconfigure systems to restore understanding of ops Goals: understand where you can 
as much capability as possible; limitations for remaining flight: go and where is “best” to go; 
understand generally what is possible re ee look at trade-offs; understand risks 
do not “fly into 


new problems 


Dynamic Integrate Airplane 
ices Capabilities with 
Prioritized Organized by Airport, Weather, 
NNC selection Phase of Flight NOTAMS, etc 
Ability to Look Identify “Compatible” 


Ahead for Limitations Airports within Range 


Thinking about Human-Autonomy Teaming Ss 


= Initially, we pull together information relevant to mission/diversion; e.g., 
"airplane compatibility / capability (range) 
"airport information 
=" weather information 


= Then, organize it in a way that flight crews can benefit, understanding 
how to present it to support collaborative decision making 


= Finally, transition some elements to a more autonomous advisor 
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Planned Activities Ss 


=" Develop a “framework/language” for communicating airplane 
capabilities 
= Pilot interviews and prototyping 


"Develop a small set of failure cases 
= Develop system models to simulate system failures 


= Collaborate with industry (e.g. SAA with Boeing) 
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Thank you 


